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General  Aviation  Pilot  Performance  Following  Unannounced  In-Flight 
Loss  of  Vacuum  System  and  Associated  Instruments  in  Simulated 
Instrument  Meteorological  Conditions 


background 

There  has  been  a  concern  with  instrument  flight 
and  loss  of  attitude  awareness  for  at  least  the  last  50 
years.  There  are  two  primary  situations  where  loss  of 
attitude  awareness  may  lead  to  a  fatal  accident.  The 
first  is  when  a  non-instrument-rated  pilot  inadvert¬ 
ently  or  intentionally  enters  instrument  meteorologi¬ 
cal  conditions  (IMC) ,  is  unable  to  maintain  the  attitude 
of  the  aircraft,  and  ultimately  enters  either  a  spiral 
dive  or  increasingly  severe  oscillations  that  ultimately 
lead  to  aircraft  structural  failure.  The  AOPA  Founda¬ 
tion,  Inc.,  funded  a  study  at  the  University  of  Illinois 
Institute  of  Aviation  that  was  reported  by  Bryan, 
Stonecipher,  and  Aron  (1954)  in  which  a  procedure 
was  developed  to  help  visual-flight-rules  (VFR)  pilots 
who  had  inadvertently  wandered  into  IMC  to  return 
to  visual  meteorological  conditions  (VMC).  Baseline 
data  were  collected  at  the  beginning  of  the  study  to 
determine  with  what  frequency  pilots  without  instru¬ 
ment  experience  would  enter  potentially  flight-termi¬ 
nating  conditions.  The  20  pilots  ranged  in  age  from 
19  to  60  years,  had  no  previous  instrument  experi¬ 
ence,  and  had  a  minimum  of  experience  with  the 
Beechcraft  Bonanza.  Total  pilot  time  ranged  from  31 
to  1625  hours.  In  their  first  exposure  to  simulated 
instrument  conditions  (created  by  wearing  blue  goggles 
in  a  cockpit  with  orange  plexiglas  covering  the  front 
and  side  windows),  19  of  the  20  entered  a  “graveyard 
spiral”  within  an  average  of  3  minutes  after  losing 
their  contact  view  of  the  outside  world.  The  20th 
placed  the  aircraft  into  a  whip-stall  attitude.  These 
results  were  obtained  with  cockpit  instrumentation 
sufficient  to  conduct  instrument-referenced  flight. 

The  second  contributing  situation  is  the  one  in 
which  instrument-rated  pilots  in  IMC  lose  their  atti¬ 
tude  reference  through  vacuum/pressure  system  or 
instrument  failure.  The  majority  of  the  207,000  air¬ 
planes  in  the  general  aviation  (GA)  fleet  have  vacuum- 
powered  attitude  indicators  (AIs)  and  heading 
indicators.  Many  of  those  same  airplanes  are  not 
equipped  with  back-up  or  secondary  attitude  indica¬ 
tors  or  a  back-up  vacuum  pump.  Therefore,  instru¬ 
ment-rated  pilots  must  demonstrate  the  ability  to  fly 
airplanes  in  “partial-panel”  (loss  ofvacuum  instruments) 


conditions  as  part  of  their  initial  and  recurrent  train¬ 
ing.  This  usually  entails  maintaining  controlled  flight 
using  indications  from  the  pitot/static  system  instru¬ 
ments  (airspeed  indicator,  vertical  speed  indicator, 
and  altimeter) ,  electric  gyro  instruments  (turn  coordi¬ 
nator),  and  magnetic  instruments  (compass).  Inas¬ 
much  as  partial-panel  flying  is  usually  simulated  by 
covering  up  the  supposedly  “failed”  instruments,  pi¬ 
lots  do  not  have  the  opportunity  to  experience  a 
realistic  vacuum  failure,  in  which  they  would  have  to 
detect  and  diagnose  the  failure  -  unless  it  is  an  actual 
emergency.  This  type  of  mechanical  failure  (vacuum 
system  or  related  instruments)  has  been  documented 
as  a  causal  factor  in  only  about  three  accidents  per 
year,  which  is  1 1%  of  all  documented  spatial  disorien¬ 
tation  accidents.  However,  these  accidents  result  in 
fatalities  approximately  90%  of  the  time  (Landsberg, 
2002;  data  from  the  Air  Safety  Foundation,  ASF, 
database  of  National  Transportation  Safety  Board 
accident  reports).  If  one  was  to  look  at  the  combina¬ 
tion  of  a  VFR  pilot  entering  IMC  and  experiencing  a 
vacuum-system  failure,  thus  losing  any  atritude  refer¬ 
ence,  it  is  not  difficult  to  imagine  the  fatality  rate 
being  even  higher  (little  data  exist,  however,  on  this 
specific  combination  of  factors).  That  is  to  say,  if 
pilots  who  flew  primarily  by  visual  reference  had 
difficulty  flying  by  reference  to  a  full  set  of  instru¬ 
ments,  it  is  likely  that  they  would  be  completely 
unable  to  continue  under  partial-panel  conditions. 

This  study  is  a  continuation  of  a  study  conducted 
for  the  AO  PA/ ASF  by  Martinez  (2000)  and  adminis¬ 
tered  by  Flight  Safety  International  (FSI)  in  2000. 
Martinez  reported  on  pilot  performance  following  the 
failure  of  an  aircraft  vacuum  system  in  single-engine 
Cessna  208  and  Cessna  210  simulators,  with  motion 
disabled.  Beringer  and  Ball  (2001)  reported  a  similar 
study  in  fixed-base  single-engine  Cessna  172  and 
Piper  Malibu  simulators,  with  results  comparable  to 
Martinez’. 

In  the  Martinez  study,  66.7  %  of  the  24  test  flights 
resulted  in  loss  of  control  and  50  %  of  the  flights 
ended  in  a  crash.  Beringer  and  Ball’s  results  from  a 
sample  of  60  pilots  showed  that  27  %  of  the  1 1  pilots 
flying  the  Malibu  with  the  electric  horizontal  situation 
indicator  (HSI)  would  have  exceeded  performance 
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limitations  of  the  aircraft  or  struck  the  ground.  A 
simulated  vacuum-driven  directional  gyro  (DG)  was 
depicted  in  place  of  the  HSI  to  represent  the  majority 
of  low-end  GA  aircraft  for  one  group,  and  83  %  of 
those  12  pilots  lost  control,  exceeded  performance 
limitations  of  the  aircraft,  or  would  have  struck  the 
ground.  When  a  back-up  AI  was  depicted  in  place  of 
the  turn  coordinator  (TC),  33  %  of  the  pilots  in  that 
group  were  unsuccessful  in  continuing  the  flight.  Best 
performance  was  obtained  with  a  back-up  AI,  HSI 
and  turn  coordinator  (only  8  %  loss).  The  Cessna  172 
pilots,  with  a  warning  flagon  the  AI,  fared  better,  with 
only  one  (8  %)  loss  of  control.  However,  differences 
in  stability  between  the  Malibu  simulator  and  the 
Cessna  simulator  (more  stable  in  roll)  placed  limita¬ 
tions  on  interpretation.  Beringer  and  Ball  recom¬ 
mended  replacing  the  DG  and  very-high-frequency 
omni  range  (VOR)  heads  with  an  HSI,  freeing  up  an 
instrument  location  for  a  back-up  electric  AI. 

The  Air  Safety  Foundation,  in  coordination  with 
the  FAA  Civil  Aerospace  Medical  Institute  (CAMI), 
developed  the  present  study  to  collect  baseline  aircraft 
data  evaluating  pilots’  skills  in  dealing  with  an  unan¬ 
nounced  vacuum  failure  in  flight  for  comparison  with 
results  obtained  in  flight  simulators.  The  following 
sections  describe  the  details  of  the  effort  and  the 
results  obtained. 


Participants 

Forty-one  volunteer  pilots  (40  males,  1  female) 
were  selected  from  approximately  300  applicants  who 
responded  to  an  announcement  on  the  ASF  Web  site. 
The  primary  goal  in  the  selection  process  was  to 
choose  a  wide  variety  of  pilots,  regarding  demograph¬ 
ics  and  flight  experience.  Pilots  participated  without 
monetary  compensation.  Table  1  presents  demo¬ 
graphic  data  for  both  the  Archer  and  the  Bonanza 
pilot  groups. 

Equipment 

Aircraft .  The  two  aircraft  used  were  a  simple  (Piper 
Archer  PA-28)  (see  Figure  1)  and  a  complex  (Beechcraft 
Bonanza  A36;  see  Figure  2)  airplane.  Each  was 
equipped  with  all  Federal  Aviation  Regulation  -  FAR 
-  required  items  for  a  single-pilot  IFR  flight.  Polarized 
material  was  placed  across  the  lower  portion  of  the 
windscreen  and  left  side  window  of  each  aircraft  (see 
Figure  3)  so  that  approximately  the  lower  two-fifths  of 
the  windscreen  was  covered.  The  F rancis  hood  used  to 
simulate  IMC  (see  inset,  Figure  3)  contained  the  same 
polarized  material  in  the  eye  openings,  oriented  90°  to 


Table  1 .  Pilot  demographic  data  by  group. 


ARCHER  (n=25  males) 

Variable  name 

Mean 

Median 

Minimum 

Maximum 

Age  (years) 

50 

53 

20 

79 

Total  Pilot-in-Command  time  (hours) 

4,358.0 

1,477 

161.0 

24,750.0 

Total  instrument  time  (hours) 

1,528.0 

283 

20.0 

14,000.0 

Total  instrument  time  in  the  last  90  days 

3.9 

3 

0.0 

9.7 

Years  since  rating  received 

19.2 

19 

0.3 

58.0 

BONANZA  (n=15  males,  1  female) 

Age  (years) 

46 

44 

32 

62 

Total  Pilot-in-Command  time  (hours) 

1,714.0 

1,397.0 

195.0 

5,495 

Total  instrument  time  (hours) 

294.0 

212.0 

45.0 

816 

Total  instrument  time  in  the  last  90  days 

4.7 

2.5 

0.0 

25 

Years  since  rating  received 

13.2 

12.9 

1.2 

32 

2 


Figure  1.  Archer  instrument  panel. 

the  windshield  material.  This  arrangement  allowed 
the  pilots  to  see  inside  the  cockpit,  but  eliminated  the 
outside  view  immediately  above  the  glare  shield. 

The  PA-28  flights  consisted  of  three  groups  (Table 
2):  (1)  Group  A  -  a  failure  of  the  AI  and  the  DG,  (2) 
Group  B  -  same  as  Group  A  but  received  30  minutes 
of  partial-panel  instruction  in  a  personal-computer- 
based  aviation  training  device  (PCATD)  prior  to  the 
flight,  and  (3)  Group  C  -  same  as  group  A  but  had  a 
failure-annunciator  light  (vacuum)  on  the  panel.  The 
A36  flights  consisted  of  two  groups  (see  Table  2):  (1) 
Group  A  -  a  failure  of  the  AI  only,  (2)  Group  B  —  a 
failure  of  the  AI  and  the  HSI. 

Data  recording .  Several  forms  of  data  were  recorded 
for  each  flight.  Flight  performance  data  were  recorded 
via  a  Cambridge  Aero  Instruments  GPS  Navigator 
and  Secure  Flight  Recorder.  This  generated  a  plan- 
view  map  and  vertical-profile  view  of  the  flight  path 
for  the  purpose  of  assessing  average  and  maximum 
flight-path  deviations.  A  color  digital  video  recording 
was  also  obtained  during  the  flight  with  audio  of  all 
intercom/radio  communications.  The  field  of  view  of 
the  camera,  which  was  attached  to  the  cabin  headliner 


Figure  2.  Bonanza  instrument  panel. 


Figure  3.  Polaroid  material  across  windscreen  of 
Piper  Archer  with  (inset)  Francis  hood. 


behind  and  to  the  right  of  the  front  left  seat,  included 
the  subject  pilot,  key  flight  instrumentation,  and  the 
forward  view  out  of  the  windscreen.  Additionally,  the 
pilot  and  evaluator  each  completed  a  post-scenario 
questionnaire  at  the  conclusion  of  the  flight  . 


Table  2.  Operational  instruments  available  for  each  pilot  group.  X  indicates  feature 
was  present. 


Pilot  Group  DG  HSI 


Archer  A 


Archer  B 


Archer  C 


Bonanza  A 
Bonanza  B 


N/A  X 


Archer  implementation  of  vacuum  failure.  Prior  to 
each  flight,  an  airframe  and  powerplant  (A&P)  me¬ 
chanic  disengaged  the  aircraft’s  engine-driven  vacuum 
system.  Therefore,  the  AI  and  DG  were  fully  opera¬ 
tional  only  via  the  standby  vacuum  system.  During 
the  flight,  the  evaluator  disengaged  the  standby  sys¬ 
tem  via  a  switch  in  the  cockpit,  thereby  failing  the 
vacuum-driven  instruments  in  a  realistic  manner. 

Bonanza  implementation  of  vacuum  failure .  Prior 
to  each  flight,  an  A&P  mechanic  disabled  the  aircraft’s 
engine-driven  instrument  air  pressure  pump.  The  AI 
was  powered  by  the  standby  system.  The  HSI  is 
electrically  powered,  so  no  maintenance  was  required 
before  the  flights  for  that  instrument.  During  the 
flight,  the  evaluator  disengaged  the  standby  air  pres¬ 
sure  system  and  HSI  via  their  individual  circuit  break¬ 
ers  in  the  cockpit.  It  is  important  to  note  that,  because 
the  AI  was  vacuum-driven  and  the  HSI  was  electric, 
this  was  not  a  “real  world”  failure.  It  would  be  rare  for 
both  vacuum  systems  and  one  electric  instrument  to 
fail  during  flight. 

Procedures  and  Tasks 

Each  scenario  began  with  a  pre-flight  interview  and 
briefing  involving  the  volunteer  pilot  and  the  evalua¬ 
tor.  Safety  information  for  the  flight  was  discussed, 
and  each  volunteer  completed  a  consent  form  and  a 
flight-experience  questionnaire.  The  pilot  was  briefed 
about  proper  aircraft  operation,  including  airspeed 
and  power  settings,  and  the  flight  plan  was  discussed. 
The  volunteers  were  told  they  would  be  evaluated  on 
their  execution  of  IFR  procedures.  The  autopilot  was 
turned  off  for  the  duration  of  each  flight. 

After  the  briefing  and  pre-flight  inspection  of  the 
airplane,  the  pilots  departed  the  Frederick  Municipal 
Airport  (FDK)  in  Frederick,  Maryland.  The  evaluator 
acted  as  an  air  traffic  controller  (ATC),  giving  the 
pilot  heading  vectors  for  the  instrument  landing  sys¬ 
tem  (ILS)  Runway  23  approach  at  FDK.  The  purpose 
of  this  first  approach  was  to  allow  the  volunteer  to 
practice  flying  in  simulated  IMC  and  to  allow  the 
flight  data  recorder  to  establish  a  baseline  for  the 
pilot’s  performance  under  normal  conditions.  The 
approach  was  discontinued  at  approximately  800  feet 
above  ground  level  (AGL).  At  that  time,  ATC  issued 
a  clearance  for  the  pilot  to  climb  to  3,000’  and  fly  a 
heading  of  270°  to  the  Eastern  WV  Regional/Shepherd 
Airport  (MRB)  in  Martinsburg,  West  Virginia. 

During  the  climb  after  the  ILS  approach,  at  a 
specific  standardized  point  (point  D  in  Figure  4),  the 
aircraft  vacuum/pressure  pump  (and  HSI  in  the  Bo¬ 
nanza)  was  disengaged  without  the  pilot  s  knowledge, 
leading  to  the  eventual  loss  of  the  aircraft’s  attitude 


and  heading  indicators.  The  pilot’s  task  was  to  main¬ 
tain  control  of  the  aircraft,  select  the  best  option(s)  to 
pursue,  navigate  accurately,  communicate  effectively 
with  ATC,  and  complete  the  flight  with  a  safe  landing 
at  either  the  destination  or  an  alternate  airport. 

The  simulated  weather  conditions  were  such  that 
FDK  was  the  best  alternate  airport.  “ATC”  provided 
vectors  to  a  point  that  provided  the  pilot  an  intercept 
heading  and  altitude  to  the  ILS  Runway  23  approach 
at  FDK.  If  requested  by  the  pilot,  “ATC”  provided 
no-gyro  vectors  above  2,500’.  No-gyro  vectors  con¬ 
sisted  of  the  direction  of  turn,  and  when  to  start  and 
stop  that  turn.  The  evaluator  took  control  of  the 
airplane  if  the  pilot  at  any  time  maneuvered  to  a  bank 
angle  approaching  60  degrees  and  increasing,  if  the 
aircraft’s  airspeed  was  approaching  Vne  (never-exceed 
speed)  and  increasing,  if  the  aircraft  was  approaching 
a  stall  condition,  or  for  any  other  reason  deemed 
necessary  for  the  safety  of  the  flight.  All  flights  were 
conducted  in  weather  conditions  that  would  allow  the 
scenario  to  be  completed  in  VFR  conditions.  The 
evaluator  acted  as  pilot  in  command  for  each  flight 
relative  to  flight  safety  issues. 

RESULTS 

Response  to  the  vacuum-failure  event  required  two 
tasks  to  be  performed.  First,  the  pilot  had  to  recognize 
that  a  failure  of  some  kind  had  occurred  and  correctly 
diagnose  it  and,  second,  the  pilot  then  had  to  success¬ 
fully  control  the  aircraft  using  the  flight  data  remain¬ 
ing.  The  following  sections  consider  each  component 
in  turn. 

Recognition  Time 

Time  to  detect/recognize  the  failure  was  measured 
from  the  time  the  failure  was  initiated  to  the  first 
verbal  report  by  the  participant  of  something  being 
“wrong.”  Although  some  pilots  attempted  adjust¬ 
ments  to  the  AI  prior  to  verbal  reporting,  the  only 
consistent  scoring  point  that  could  be  used  was  the 
verbal  report. 

The  PA-28  pilots  averaged  a  higher  recognition 
time,  with  an  average  of  6.9  minutes  for  the  entire 
group.  (See  Table  3.)  The  pilots  who  flew  partial- 
panel  on  the  PCATD  prior  to  their  flight  recognized 
the  instrument  failure  more  quickly;  however,  the 
differences  among  the  Archer  groups  did  not  attain 
statistical  significance  [F  (2,21)=2.54,  p>0.1]  (4.9 
minutes,  vs.  7.6  for  the  other  groups).  Neither  can 
comparisons  be  made  between  the  two  aircraft  because 
of  potential  differences  in  the  rate  at  which  the  vacuum/ 
pressure-driven  instruments  in  each  failed.  The 
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Figure  4.  Scenario  chart  showing  point  (D)  where  vacuum/pressure  system  was  disengaged 
and  intended  flight  paths. 


Bonanza  pilots  who  experienced  a  failure  of  the  HSI 
as  well  as  the  AI  recognized  the  failure  in  an  average  of 
2.6  minutes,  which  was  significantly  faster  than  the 
average  4.6  minutes  for  those  pilots  who  had  only  an 
AI  failure  [F  (1,14)=6.372,  pc.  05].  The  HSI  was 
equipped  with  a  warning  flag  to  announce  instrument 
failure,  and  this  undoubtedly  aided  pilots  in  the 
Bonanza  B  group. 

Flight  Performance  Data 

Outcomes  of  all  flights  were  categorized  as  follows: 

Category  1:  The  pilot  had  no  problem  controlling 
the  aircraft.  The  deviation  was  less  than  20  degrees 
and  200  feet. 

Category  2:  The  aircraft  remained  under  the  pilot’s 
control,  but  with  more  effort  than  the  category  1 
pilots.  The  deviation  was  between  20  and  40  degrees 
inclusive  and  200  to  400  feet. 

Category  3:  The  aircraft  was  barely  under  control 
-  the  pilot  was  struggling  significantly. 

Category  4:  The  evaluator  had  to  take  control  of 
the  aircraft.  Had  this  been  a  real  instrument  failure  in 
IMC,  the  flight  likely  would  have  resulted  in  a  crash. 


Archer.  All  of  the  PA-28  pilots  were  able  to  main¬ 
tain  control  of  the  aircraft  under  partial-panel  condi¬ 
tions.  However,  some  became  disoriented  and  were 
not  able  to  successfully  execute  an  approach  and 
landing  to  the  airport.  Thirty-two  percent  of  the 
pilots  did  not  successfully  complete  the  approach 
(i.e.,  68  %  were  successful).  An  example  of  the  hori¬ 
zontal  and  vertical-profile  plots  obtained  from  each 
flight  is  depicted  in  Figure  5. 

Archer  Procedures .  Upon  noticing  the  vacuum  sys¬ 
tem  failure,  28  percent  of  the  PA-28  pilots  declared  an 
emergency  to  “ATC”  (the  experimenter).  At  the  next 
level  of  urgency,  68  %  notified  ATC  of  the  problem 
without  declaring  an  emergency,  but  one  pilot  (4  %) 
gave  “ATC”  no  notification  of  the  problem.  Distrac¬ 
tion  played  a  significant  role  in  that  28  %  of  the  pilots 
covered  the  failed  instruments  to  prevent  distraction 
or  being  mislead  by  the  now-failed  indicators.  The 
remaining  72  %  did  not  do  so  and  tended  to  include 
the  failed  instruments  in  their  scan,  indicating  to  the 
experimenter  that  this  was  a  distracting  situation. 

Bonanza.  Twenty-five  percent  of  the  A36  pilots  lost 
control  of  the  aircraft.  All  four  of  those  pilots  (3  males, 
1  female)  were  in  Bonanza  Group  B  (loss  of  AI  and 
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Table  3.  Mean  recognition  time  per  pilot  group. 


Pilot  Group 

Average  time  to  recognize  failure 
(minutes) 

Archer  A 

7.3 

Archer  B 

4.9 

Archer  C 

7.9 

Bonanza  A 

4.6* 

Bonanza  B 

2.6* 

Note: 

*  The  only  significant  difference  was  between  these  means.  Archer  A  pilots 
used  standard  Archer  instrumentation. 

Archer  B  pilots  received  30  minutes  of  partial  panel  PCATD  training  prior 
to  their  flight. 

Archer  C  pilots  had  a  vacuum  annunciator  light  as  a  warning  when  the 
vacuum  system  failed. 

Bonanza  A  pilots  experienced  an  AI  failure. 

Bonanza  B  pilots  experienced  both  AI  and  HSI  failures. 


HSI).  Two  of  these  four  were  experienced 
Bonanza  pilots,  thus  the  effect  is  not  solely 
attributable  to  lack  of  familiarity  with  the 
aircraft  type.  Sixty-nine  percent  of  the  pilots 
were  able  to  successfully  complete  a  partial- 
panel  approach,  including  two  of  the  pilots 
who  had  lost  aircraft  control  but  were  given  a 
second  chance  to  fly  the  aircraft  partial  panel. 

Bonanza  procedures .  None  of  the  A3 6  pi¬ 
lots  declared  an  emergency  to  “ATC,”  choos¬ 
ing  instead  to  simply  notify  “ATC”  of  the 
problem  and  request  assistance.  One  pilot  (6 
%)  covered  the  failed  instruments  -  that  pilot 
had  no  problem  controlling  the  aircraft,  and 
is  classified  as  a  Category  1  flight.  Flight 
performance  outcomes  for  each  group  are 
summarized  in  Table  4. 

The  distribution  of  loss-of-control  flights 
was  such  that  a  chi-square  analysis  was  not 
appropriate  given  the  number  of  cells  having 
expected  and/or  observed  frequencies  less  than 
five.  The  distribution  of  failed  approaches  by 
aircraft  type  was  amenable  to  such  an  analy¬ 
sis,  but  there  was  no  significant  effect  attributable  to 
aircraft  type. 

Response  to  Warning  Indicators/instruments 

Only  one  (12.5  %)  of  the  Archer  pilots  in  GroupC 
(vacuum  annunciator  light  available)  actually  noticed 
the  light.  Only  one  (4  %)  out  of  all  of  the  Archer  pilots 
noticed  the  vacuum  gauge  at  the  onset  of  the  emer¬ 
gency.  The  others  simply  used  it  to  verify  the  system 
failure  once  the  instruments  were  tumbled.  Previous 
studies  (Beringer  and  Ball,  2001)  listed  the  vacuum- 
low  annunciator  light  as  one  of  the  first  failure  indi¬ 
cations  detected  by  the  participants  in  the  conditions 
where  it  was  available.  The  present  finding  is  not  too 


surprising,  given  that  the  pilots  were  wearing  the 
Francis  hood,  which  greatly  limits  peripheral  and 
parafoveal  vision,  and  that  the  vacuum  gauge  was 
located  on  the  right  side  of  the  cockpit,  well  out  of  the 
area  of  vision  when  viewing  the  primary  instrument 
cluster.  It  should  also  be  noted  that  the  vacuum 
annunciator  light  was  small  and  not  very  bright.  This 
suggests  that  some  attention  may  need  to  be  given  to 
indicator  placement  and  conspicuity. 

Pilot  Experience  Variables 

Several  questionnaire  forms  were  administered  to 
gather  experience  data  from  each  pilot.  Questions 
assessed  experience  with  the  specific  model  of  aircraft 

to  be  flown,  certificates/ 
ratings,  date  of  instru¬ 
ment  rating,  pilot-in- 
command  hours  (total 
and  last  90  days),  in¬ 
strument  hours  (total, 
last  12  months,  last  90 
days,  and  “actual”),  in¬ 
strument  training  in  the 
last  90  days  (last  date  of 
partial-panel  training, 
amount  of  partial-panel 
training,  description  of 
that  training),  hours 
flown  annually,  number 
of  approaches  in  the  last 


Table  4.  Frequency  of  outcome  by  aircraft  group.  Subscripts  indicate 
number  of  pilots  in  that  category  who  failed  to  complete  the  instrument 
approach  in  partial-panel  conditions. 


Pilot  Group 

Category  1: 

Successful 
partial  panel 

Category  2: 
Required 
more  effort 

Category  3: 

Barely 

controlled 

Category  4: 

Safety  pilot 
took  over 

Total 

sample  size 

Archer  A 

6, 

52 

i 

0 

12 

Archer  B 

32 

2, 

0 

0 

5 

Archer  C 

4 

4-2 

0 

0 

8 

Bonanza  A 

1 

2 

1, 

0 

4 

Bonanza  B 

4 

2, 

2, 

4-2 

12 

6 


Figure  5.  Example  plan-view  (top)  and  vertical  profile  (bottom)  plots  of  a  flight. 
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90  days,  and  experience  using  GPS  equipment.  Only 
two  observations  are  worth  noting.  The  PA-28  pilots 
who  had  more  than  three  instrument  flight  hours 
during  the  90  days  preceding  their  flight  (40  %)  were 
noticeably  more  proficient.  Nine  (75  %)  flew  Cat¬ 
egory  1  flights  and  three  (25  %)  flew  category  2 
flights.  However,  it  should  be  noted  that  no  signifi¬ 
cant  correlations  were  found  between  pilot  experience 
variables  and  performance  variables,  including  cat¬ 
egorization,  for  the  PA-28  sample.  This  is  undoubt¬ 
edly  due  to  the  small  sample  size.  The  only 
pilot-experience  variable  that  showed  a  significant 
correlation  with  performance  (Spearman’s  rho)  was 
total  pilot-in-command  (PIC)  hours  (rho  =  -.622, 
p=.0 1).  Practically  speaking,  a  higher  PIC  total  was 
associated  with  a  greater  likelihood  of  obtaining  a 
better  (lower-numbered)  category  of  performance. 

Simulator  and  Training  Benefits 

Pilots  reported  that  flying  the  partial-panel  trial 
was  a  beneficial  experience  and  that  they  felt  better 
prepared  to  handle  this  type  of  emergency  situation  in 
actual  IMC  after  having  participated  in  the  study. 
Those  who  flew  the  Elite  PCATD  flight  simulator 
prior  to  their  aircraft  flights  indicated  that  the  train¬ 
ing  helped  them.  Specifically,  they  said  that  they  were 
better  prepared  to  handle  an  emergency  and  were 
already  “warmed  up”  to  fly  after  the  training-device 
practice.  Recommendations  related  to  training  in¬ 
cluded  encouraging  flight  schools  to  have  at  least  one 
aircraft  configured  to  present  vacuum-system  failures 
in  flight,  providing  students  with  the  opportunity  to 
detect  and  diagnose  this  failure  in  a  realistic-onset 
environment.  This  was  proposed  in  contrast  with  the 
present  practice  of  having  the  instructor  place  covers 
over  instruments  to  be  deemed  “failed”  during  prac¬ 
tice.  In  addition,  it  was  the  majority  opinion  that 
pilots  need  more  practice  flying  with  partial-panel 
instrumentation. 

Limitations  on  Interpretation  of  the  Data 

A  number  of  factors  could  have  affected  the  out¬ 
come  of  the  airborne  trials  and  should  be  considered 
when  interpreting  the  data.  Each  is  listed  and  dis¬ 
cussed  briefly  below: 

Artificial  termination  of  flight:  The  evaluator  termi¬ 
nated  any  flight  at  the  onset  of  unsafe  conditions. 
Therefore,  it  was  not  possible  to  determine  if  a  crash 
would  have  actually  occurred.  For  the  four  A36  losses  of 
control,  it  is  likely  that  the  flight  would  have  been  fatal. 


Source  of  participants:  As  volunteers,  the  pilots 
may  not  have  been  an  accurate  representation  of  the 
309,000  instrument-rated  pilots  in  the  US.  It  is  likely 
that  volunteers  were  more  proficient  than  average 
general-aviation  pilots. 

Experience  with  test  aircraft:  The  pilots  had  no 
flight  time  in  the  actual  study  aircraft  (8121K  and 
7236W)  prior  to  the  experiment.  However,  most 
pilots  are  familiar  with  PA-28  aircraft  and  have  flown 
them  as  part  of  their  initial  flight  training.  That  may 
be  why  a  higher  percentage  of  the  PA-28  pilots  were 
able  to  control  the  aircraft  during  the  loss  of  horizon 
reference  as  compared  with  the  Bonanza  groups. 

Simulated  IMC  and  safety  pilot:  The  scenario  was 
simulated  single  pilot  IMC,  actually  flown  under 
VFR  conditions  with  a  safety  pilot.  The  pilots  likely 
would  perform  differently  if  they  had  actually  been 
alone  during  a  real  emergency  (more  may  have  de¬ 
clared  an  emergency;  fewer  may  have  pressed  the 
limits  of  the  performance  envelope). 

Compound  systems  failure,  Bonanza:  The  A36 
had  a  pressure-air-powered  attitude  indicator  (AI), 
but  an  electric  horizontal  situation  indicator  (HSI). 
Therefore,  there  was  not  an  actual  pressure-air-system 
failure,  but  instead,  two  individual  instrument  fail¬ 
ures.  That  may  have  resulted  in  more  confusion  for 
the  pilots  flying  the  scenario  who  were  familiar  with 
redundant  systems,  although  none  of  the  A36  pilots 
mentioned  this  issue  during  the  post-flight  briefing. 

Supplemental  visual  cues:  Pilots  may  have  re¬ 
ceived  some  supplemental  cues  to  aircraft  attitude 
from  shadows  playing  across  the  cockpit  or  from 
occasional  fragmentary  outside-world  views  when  the 
head  was  tilted  (polarizing  fields  not  fully  orthogo¬ 
nal).  This  could  have  improved  their  performance 
slightly  as  a  group. 

CONCLUSIONS  AND 
RECOMMENDATIONS 

The  results  of  this  study  reflect  to  some  degree  the 
simulator  data  obtained  by  Beringer  and  Ball  (2001). 
On  one  hand,  the  results  with  the  simple  airplanes 
(Cessna  172  and  Piper  PA-28)  were  very  similar  in 
that  pilots  flying  that  simulator  and  aircraft  were 
largely  in  control  of  the  flight  to  completion.  It  was 
also  true  that  virtually  all  of  the  recorded  losses  of 
control  occurred  with  the  complex  aircraft  (Malibu 
simulator  and  Bonanza  A-36  aircraft;  only  one  loss 
was  recorded  in  the  Skyhawk  simulation).  These  re- 
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suits,  taken  together,  suggest  that  we  are  likely  to  see 
more  problems  associated  with  high-performance  air¬ 
craft  than  with  the  simpler  fixed-gear,  fixed-pitch- 
propeller  aircraft  when  vacuum  failures  are 
encountered.  However,  it  should  also  be  noted  that 
the  loss  rate  using  the  airborne  platform  (Bonanza) 
was  far  lower,  for  equivalent  instrument  equipage, 
than  that  found  in  the  Malibu  simulator.  Given  the 
factors  listed  that  may  have  compromised,  to  some 
degree,  the  outcomes  of  the  airborne  observations  and 
the  simulator  trials,  we  believe  that  the  likely  rate  at 
which  serious  controlling  difficulties  may  be  encoun¬ 
tered  in  actual  aircraft  lies  somewhere  between  the 
two  figures. 

Unfortunately,  no  airborne  assessment  of  the  effi¬ 
cacy  of  a  back-up  attitude  indicator  was  obtained. 
Nonetheless,  it  is  reasonable  to  believe,  based  upon 
the  simulator  data,  that  the  percentage  of  complex- 
airplane  pilots  experiencing  serious  difficulties  after 
loss  of  the  horizon  reference  could  be  further  reduced 
with  the  proper  instrumentation.  The  presence  of  the 
functioning  electric  HSI  after  the  vacuum  failure 
appeared  to  greatly  reduce  the  difficulty  of  maintain¬ 
ing  control  in  the  Bonanza,  and  this  was  consistent 
with  data  from  the  Beringer  and  Ball  simulator  study. 

Given  the  results  to  date,  as  represented  in  the  cited 
simulator  studies  and  the  airplane  study  reported 
herein,  the  following  recommendations  are  submitted 
for  consideration: 

Training:  Attention  should  be  given  to  any  train¬ 
ing  wherein  detection  and  diagnosis  can  be  practiced 
in  as  realistic  a  setting  as  possible.  This  should  include 
simulators,  aviation  training  devices,  and/or  aircraft 
with  vacuum  systems  specially  adapted  for  the  re¬ 
moval  of  power  from  at  least  one  attitude  instrument 
on  the  trainee’s  side. 

Warnings  indicators:  Attention  should  be  given  to 
the  location  and  design  of  indicators  for  system  or 
instrument  failure.  Vacuum-failure  indicators  should 
be  located  as  close  to  the  primary  panel  as  possible  and 
should  be  very  conspicuous.  Attitude  indicators  should 
be  flagged  to  indicate  loss  of  power,  whether  electrical 
or  air.  This  can  also  be  done  by  referencing  gyro  RPM 
and  indicating  whenever  it  drops  below  the  minimum 
deemed  necessary  for  reliable  operation. 

Instrumentation:  Three  levels  of  instrumentation 
modification  are  suggested  in  order  of  increasing  cost/ 
complexity: 


1)  Given  that  a  reliable  heading  reference  has 
been  demonstrated  to  be  useful  in  maintaining  con¬ 
trol,  one  option  is  to  replace  the  directional  gyro  with 
a  heading  indicator  that  is  independent  of  the  attitude 
indicator’s  power  source  and  that  will  continue  to 
function,  obviating  the  need  for  referencing  a  fluid- 
filled  compass.  This  also  brings  the  heading  indicator 
into  the  primary  panel/scan  and  reduces  both  scan¬ 
ning  requirements  and  the  need  to  compensate  for  the 
known  behaviors  of  a  fluid-filled  compass. 

2)  Integrating  instrumentation  can  also  produce 
decluttering  benefits  and  free  up  space  for  additional 
instrumentation.  Thus,  another  option  is  to  install  an 
HSI  in  place  of  a  directional  gyro  and  a  VOR  head, 
providing  multiple  indications  within  one  instru¬ 
ment,  reducing  required  scan,  and  opening  up  an 
instrument  location  on  the  panel. 

3)  It  is  also  likely  that  a  back-up  attitude  indicator 
would  be  useful  in  reducing  pilot  workload  based  upon 
the  simulator-study  results.  This  must  be  independently 
powered  relative  to  the  primary  attitude  indicator,  and 
each  should  be  flagged  to  indicate  instrument  or  power- 
system  failure.  This  option,  however,  is  not  directly 
addressed  by  data  in  this  study. 
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